Introduction to Algorithms, Second Edition

problemsTo understand the class of polynomial-time solvable problems, we must first have a formalnotion of what a "problem" is. We define an abstract problem Q to be a binary relation on aset I of problem instances and a set S of problem solutions. For example, an instance forSHORTEST-PATH is a triple consisting of a graph and two vertices. A solution is a sequenceof vertices in the graph, with perhaps the empty sequence denoting that no path exists. Theproblem SHORTEST-PATH itself is the relation that associates each instance of a graph andtwo vertices with a shortest path in the graph that connects the two vertices. Since shortestpaths are not necessarily unique, a given problem instance may have more than one solution. This formulation of an abstract problem is more general than is required for our purposes. Aswe saw above, the theory of NP-completeness restricts attention to decision problems: thosehaving a yes/no solution. In this case, we can view an abstract decision problem as a functionthat maps the instance set I to the solution set {0, 1}. For example, a decision problem relatedto SHORTEST-PATH is the problem PATH that we saw earlier. If i = G, u, v, k is an instance of the decision problem PATH, then PATH(i) = 1 (yes) if a shortest path from u to vhas at most k edges, and PATH(i) = 0 (no) otherwise. Many abstract problems are notdecision problems, but rather optimization problems, in which some value must be minimizedor maximized. As we saw above, however, it is usually a simple matter to recast anoptimization problem as a decision problem that is no harder. EncodingsIf a computer program is to solve an abstract problem, problem instances must be representedin a way that the program understands. An encoding of a set S of abstract objects is a mappinge from S to the set of binary strings. For example, we are all familiar with encoding thenatural numbers N = {0, 1, 2, 3, 4,...} as the strings {0, 1, 10, 11, 100,...}. Using thisencoding, e(17) = 10001. Anyone who has looked at computer representations of keyboardcharacters is familiar with either the ASCII or EBCDIC codes. In the ASCII code, theencoding of A is 1000001. Even a compound object can be encoded as a binary string bycombining the representations of its constituent parts. Polygons, graphs, functions, orderedpairs, programs-all can be encoded as binary strings. Thus, a computer algorithm that "solves" some abstract decision problem actually takes anencoding of a problem instance as input. We call a problem whose instance set is the set ofbinary strings a concrete problem. We say that an algorithm solves a concrete problem in timeO(T (n)) if, when it is provided a problem instance i of length n = |i|, the algorithm canproduce the solution in O(T (n)) time. A concrete problem is polynomial-time solvable,therefore, if there exists an algorithm to solve it in time O(n) for some constant k. We can now formally define the complexity class P as the set of concrete decision problemsthat are polynomial-time solvable.We can use encodings to map abstract problems to concrete problems. Given an abstractdecision problem Q mapping an instance set I to {0, 1}, an encoding e : I → {0, 1}* can beused to induce a related concrete decision problem, which we denote by e(Q). If the solutionto an abstract-problem instance i I is Q(i) {0, 1}, then the solution to the concrete-problem instance e(i) {0, 1}* is also Q(i). As a technicality, there may be some binarystrings that represent no meaningful abstract-problem instance. For convenience, we shallassume that any such string is mapped arbitrarily to 0. Thus, the concrete problem producesthe same solutions as the abstract problem on binary-string instances that represent theencodings of abstract-problem instances. We would like to extend the definition of polynomial-time solvability from concrete problemsto abstract problems by using encodings as the bridge, but we would like the definition to beindependent of any particular encoding. That is, the efficiency of solving a problem shouldnot depend on how the problem is encoded. Unfortunately, it depends quite heavily on theencoding. For example, suppose that an integer k is to be provided as the sole input to analgorithm, and suppose that the running time of the algorithm is Θ(k). If the integer k isprovided in unary-a string of k 1's-then the running time of the algorithm is O(n) on length-ninputs, which is polynomial time. If we use the more natural binary representation of theinteger k, however, then the input length is n = ⌊lg k⌋ + 1. In this case, the running time of thealgorithm is Θ (k) =Θ(2), which is exponential in the size of the input. Thus, depending onthe encoding, the algorithm runs in either polynomial or superpolynomial time. The encoding of an abstract problem is therefore quite important to our under-standing ofpolynomial time. We cannot really talk about solving an abstract problem without firstspecifying an encoding. Nevertheless, in practice, if we rule out "expensive" encodings suchas unary ones, the actual encoding of a problem makes little difference to whether theproblem can be solved in polynomial time. For example, representing integers in base 3instead of binary has no effect on whether a problem is solvable in polynomial time, since aninteger represented in base 3 can be converted to an integer represented in base 2 inpolynomial time.We say that a function f : {0, 1}* → {0,1}* is polynomial-time computable if there exists apolynomial-time algorithm A that, given any input x {0, 1}*, produces as output f (x). Forsome set I of problem instances, we say that two encodings e1 and e2 are polynomially relatedif there exist two polynomial-time computable functions f12 and f21 such that for any i I , wehave f12(e1(i)) = e2(i) and f21(e2(i)) =e1(i). That is, the encoding e2(i) can be computed fromthe encoding e1(i) by a polynomial-time algorithm, and vice versa. If two encodings e1 and e2of an abstract problem are polynomially related, whether the problem is polynomial-timesolvable or not is independent of which encoding we use, as the following lemma shows.Lemma 34.1 Let Q be an abstract decision problem on an instance set I , and let e1 and e2 be polynomiallyrelated encodings on I . Then, e1(Q) P if and only if e2(Q) P.Proof We need only prove the forward direction, since the backward direction is symmetric.Suppose, therefore, that e1(Q) can be solved in time O(nk) for some constant k. Further,suppose that for any problem instance i, the encoding e1(i) can be computed from theencoding e2(i) in time O(n) for some constant c, where n = |e2(i)|. To solve problem e2(Q), oninput e2(i), we first compute e1(i) and then run the algorithm for e1(Q) on e1(i). How long doesthis take? The conversion of encodings takes time O(n), and therefore |e1(i)| = O(n), sincethe output of a serial computer cannot be longer than its running time. Solving the problem one1(i) takes timeO(|e1(i)|) = O(n), which is polynomial since both c and k are constants. Thus, whether an abstract problem has its instances encoded in binary or base 3 does notaffect its "complexity," that is, whether it is polynomial-time solvable or not, but if instancesare encoded in unary, its complexity may change. In order to be able to converse in anencoding-independent fashion, we shall generally assume that problem instances are encodedin any reasonable, concise fashion, unless we specifically say otherwise. To be precise, weshall assume that the encoding of an integer is polynomially related to its binaryrepresentation, and that the encoding of a finite set is polynomially related to its encoding as alist of its elements, enclosed in braces and separated by commas. (ASCII is one such encodingscheme.) With such a "standard" encoding in hand, we can derive reasonable encodings ofother mathematical objects, such as tuples, graphs, and formulas. To denote the standardencoding of an object, we shall enclose the object in angle braces. Thus, G denotes thestandard encoding of a graph G. As long as we implicitly use an encoding that is polynomially related to this standardencoding, we can talk directly about abstract problems without reference to any particularencoding, knowing that the choice of encoding has no effect on whether the abstract problemis polynomial-time solvable. Henceforth, we shall generally assume that all problem instancesare binary strings encoded using the standard encoding, unless we explicitly specify thecontrary. We shall also typically neglect the distinction between abstract and concreteproblems. The reader should watch out for problems that arise in practice, however, in whicha standard encoding is not obvious and the encoding does make a difference. A formal-language frameworkOne of the convenient aspects of focusing on decision problems is that they make it easy touse the machinery of formal-language theory. It is worthwhile at this point to review somedefinitions from that theory. An alphabet Σ is a finite set of symbols. A language L over Σ isany set of strings made up of symbols from Σ. For example, if Σ = {0, 1}, the set L = {10, 11,101, 111, 1011, 1101, 10001,...} is the language of binary representations of prime numbers.We denote the empty string by ε, and the empty language by Ø. The language of all stringsover Σ is denoted Σ*. For example, if Σ = {0, 1}, then Σ* = {ε, 0, 1, 00, 01, 10, 11, 000,...} isthe set of all binary strings. Every language L over Σ is a subset of Σ*. There are a variety of operations on languages. Set-theoretic operations, such as union andintersection, follow directly from the set-theoretic definitions. We define the complement ofL by. The concatenation of two languages L1 and L2 is the languageL = {x1x2 : x1 L1 and x2 L2}.The closure or Kleene star of a language L is the languageL*= {ε} L L L ···,where Lk is the language obtained by concatenating L to itself k times.From the point of view of language theory, the set of instances for any decision problem Q issimply the set Σ*, where Σ = {0, 1}. Since Q is entirely characterized by those probleminstances that produce a 1 (yes) answer, we can view Q as a language L over Σ = {0, 1},where L = {x Σ*: Q(x) = 1}. For example, the decision problem PATH has the corresponding languagePATH = { G, u, v, k : G = (V, E) is an undirected graph, u, v V, k ≥ 0 is an integer, andthere exists a path from u to v in G consisting of at most k edges}. (Where convenient, we shall sometimes use the same name-PATH in this caseto refer toboth a decision problem and its corresponding language.) The formal-language framework allows us to express the relation between decision problemsand algorithms that solve them concisely. We say that an algorithm A accepts a string x {0,1}* if, given input x, the algorithm's output A(x) is 1. The language accepted by an algorithm A is the set of strings L = {x {0, 1}*: A(x) = 1}, that is, the set of strings that the algorithmaccepts. An algorithm A rejects a string x if A(x) = 0.Even if language L is accepted by an algorithm A, the algorithm will not necessarily reject astring x ∉ L provided as input to it. For example, the algorithm may loop forever. A languageL is decided by an algorithm A if every binary string in L is accepted by A and every binarystring not in L is rejected by A. A language L is accepted in polynomial time by an algorithmA if it is accepted by A and if in addition there is a constant k such that for any length-n stringx L, algorithm A accepts x in timeO(n). A language L is decided in polynomial time by analgorithm A if there is a constant k such that for any length-n string x {0, 1}*, the algorithmcorrectly decides whether x L in timeO(n). Thus, to accept a language, an algorithm needonly worry about strings in L, but to decide a language, it must correctly accept or reject everystring in {0, 1}*. As an example, the language PATH can be accepted in polynomial time. One polynomial-time accepting algorithm verifies that G encodes an undirected graph, verifies that u and v arevertices in G, uses breadth-first search to compute the shortest path from u to v in G, and thencompares the number of edges on the shortest path obtained with k. If G encodes anundirected graph and the path from u to v has at most k edges, the algorithm outputs 1 andhalts. Otherwise, the algorithm runs forever. This algorithm does not decide PATH, however,since it does not explicitly output 0 for instances in which the shortest path has more than kedges. A decision algorithm for PATH must explicitly reject binary strings that do not belongto PATH. For a decision problem such as PATH, such a decision algorithm is easy to design:instead of running forever when there is not a path from u to v with at most k edges, it outputs0 and halts. For other problems, such as Turing's Halting Problem, there exists an acceptingalgorithm, but no decision algorithm exists. We can informally define a complexity class as a set of languages, membership in which isdetermined by a complexity measure, such as running time, of an algorithm that determineswhether a given string x belongs to language L. The actual definition of a complexity class issomewhat more technical-the interested reader is referred to the seminal paper by Hartmanisand Stearns [140].Using this language-theoretic framework, we can provide an alternative definition of thecomplexity class P: P = {L {0, 1}* : there exists an algorithm A that decides L in polynomial time} . In fact, P is also the class of languages that can be accepted in polynomial time.